Riemannian Tensor Completion with Side Information

Riemannian optimization methods have shown to be both fast and accurate in recovering a large-scale tensor from its incomplete observation. However, in almost all recent Riemannian tensor completion methods, only low rank constraint is considered. Another important fact, side information or features, remains far from exploiting within the Riemannian optimization framework. In this paper, we explicitly incorporate the side information into a Riemannian minimization model. Specifically, a feature-embedded objective is designed to substantially reduce the sample complexity. For such a Riemannian optimization, a particular metric can be constructed based on the curvature of the objective, which leads to a novel Riemannian conjugate gradient descent solver. Numerical experiments suggest that our solver is more efficient than the state-of-the-art when a highly accurate solution is required. Introduction Tensor completion, which aims to recover a multidimensional array or tensor from its linear measurements, is ubiquitous in machine learning applications. For example, in video inpainting, one wants to interpolate the whole video based on its partial observations pixels. (Liu et al. 2013; Li et al. 2015); in context-aware recommendation system, one wishes to predicted the unknown ratings based on the historical records (Hidasi and Tikk 2012; Liu, Wu, and Wang 2015); in multilinear and multitask learning, one intends to learn a weight tensor that maps the multidimensional predictors to the responses. (Romera-Paredes et al. 2013; Yu and Liu 2016). Tensor completion can be encapsulated by a variety of optimization problems. Amongst them, the convex models that formulate the completion task as a tensor nuclear norm penalized regression problem is most popular and wellunderstood (Wimalawarne, Sugiyama, and Tomioka 2014; Liu et al. 2013; Tomioka, Hayashi, and Kashima 2010). Many research results have proved that the convex model is guaranteed to recover the partial observed tensor under suitable assumptions (Romera-Paredes and Pontil 2013; Yuan and Zhang 2015). However, application of these models to a large-scale instance is difficult, since solving them Corresponding author Copyright c © 2017, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. involves iterative Singular Value Decomposition (SVD) of multiple huge matrices, which may be computationally prohibitive (Gandy, Recht, and Yamada 2011). Besides the convex models, tensor completion can also be modeled by nonconvex regression constrained on Riemannian manifolds (Kressner, Steinlechner, and Vandereycken 2014; Kasai and Mishra 2016) that can be solved by the Riemannian optimization framework (Absil, Mahony, and Sepulchre 2009). Empirical comparisons have shown that, in contrast to the nuclear norm solvers, Riemannian solvers use significantly less CPU time to recover the underlying tensor (Kasai and Mishra 2016). This is so because they have much cheaper per-iteration cost as SVD of huge matrices are avoided. Thus, they are more scalable to massive data. However, to accurately recover the underlying tensor, the Riemannian models demand numerous training samples. Otherwise, the Riemannian models would overfit seriously to the training set, resulting in poor performance over the test set. In matrix completion both theoretical and empirical evidence have shown that exploiting side information is helpful in reducing the sample complexity and improving the accuracy (Xu, Jin, and Zhou 2013; Chiang, Hsieh, and Dhillon 2015). Although matrix completion can be regarded as a special case of tensor completion, directly extending these models to general Riemannian tensor completion is hardly feasible, because they contains non-smooth nuclear norms in the objective functions which can not be solved in the Riemannian optimization framework (Absil, Mahony, and Sepulchre 2009). In this paper, a novel optimization model for tensor completion is proposed to explicitly incorporate the side information. Then, a novel Riemannian metric is constructed from the second-order derivative of the proposed objective function. Such metric induces an adaptive preconditioner that can accelerate the convergence speed of solvers for the proposed model. Finally, an efficient algorithm for the proposed model is derived based on Riemannian conjugate gradient descent method (Sato and Iwai 2015). As far as we know, this is the first attempt to integrate the side information in a Riemannian tensor completion model. Experimental results show that our solver outperforms stateof-the-art Riemannian solvers both in accuracy and speed. Notations and Preliminaries We use bold capital letters to denote a matrix, e.g. X. The column space of matrix X is denoted by span(X). We use Euler script to denote a linear subspace, e.g. A. A tensor is denoted by bold Euler script, e.g. X . The (i1, i2, i3)th element of tensor X is denoted by Xi1,i2,i3 . Tensor Frobenius norm is denoted by ‖ · ‖F where ‖X‖F =